1. Field of the Invention
The present invention relates to an encoded data selection method and an encoded data setting method corresponding to the above method as well as to devices of these methods. Further, the present invention relates to a recoded data creation method including an encoded data selection method and a device corresponding to the method. Further, the present invention relates to a recoding method including an encoded data setting method and a device corresponding to the method. For example, the encoded data selection method is a method of selecting a part of encoded data output from a decoding device.
2. Description of the Related Art
In an tandem-connected MPEG-2 recoding device that uses a decoded image of MPEG-2 bit stream output from an MPEG-2 decoding device as an input, there is a technology for transmitting encoded data in a bit stream after it is multiplexed to decoded image (refer to “SMPTE 319M-2000” (authenticated by SMPTE on Jan. 20, 2000)).
In the MPEG-2 recoding device that supports SMPTE 319M-2000 Standard, encoded data multiplexed to decoded data is separated therefrom and recoding is carried out making use of the separated encoded data, thereby deterioration of image quality caused by recoding can be suppressed as much as possible. Encoded data referred to here is data obtained by returning code word, which is subjected to variable length coding in a bit stream, to an original numerical value.
In SMPTE 319M-2000 Standard, since it is standardized to superimpose the encoded data to LSB of chrominance of input decoded image, encoded data of 256 bits can be transmitted by an image of one macroblock.
Referring to Table 2 of “SMPTE 327M-2000” (authenticated by SMPTE on Jan. 20, 2000) that is MPEG-2 encoded data standard, since encoded data per one macroblock of MPEG-2 is 113 bits, it can be sufficiently transmitted in a band of 256 bits per one macroblock.
Recently, attention is paid to H.264 (ISO/IEC 14496-10). There is a case in which it is required to transform H.264 bit stream encoded by a certain bit rate to H.264 bit stream having a desired bit rate or to MPEG-2 bit stream having a desired bit rate. To cope with this case, it is necessary to connect an H.264 decoding device to an H.264 recoding device or an MPEG-2 recoding device in tandem and to transmit decoded image of H.264 bit stream therebetween. Further, to suppress deterioration of image quality as much as possible, it is necessary to transmit encoded data after it is multiplexed to decoded image of an H.264 bit stream likewise SMPTE 319-2000.
Incidentally, the encoded data of H.264 per one macroblock is different depending on a coding mode of the macroblock.
The coding mode is classified to an intra-mode of in picture coding and an inter-mode of inter picture coding.
Further, the macroblock has a size of 16×16, and as shown FIGS. 1A to 1D, the macroblock is classified to four types in the inter-mode. As shown in FIG. 1A, in a first macroblock type, one macroblock is composed of one submacroblock of 16×16 pixels. As shown in FIG. 1B, in a second macroblock type, one macroblock is composed of two submacroblocks of 16×8 pixels. As shown in FIG. 1C, in a third macroblock type, one macroblock is composed of two submacroblocks of 8×16 pixels. As shown in FIG. 1D, in a fourth macroblock type, one macroblock is composed of four submacroblocks with 8×8 pixels.
Further, the respective submacroblocks in the four macro types are classified to four kinds of types. As shown in FIG. 2A, in a first submacroblock type, one submacroblock is composed of one secondary submacroblock with 8×8 pixels. As shown in FIG. 2B, in a second submacroblock type, one submacroblock is composed of two secondary submacroblocks of 8×4 pixels. As shown in FIG. 2C, in a third submacroblock type, one submacroblock is composed of two secondary submacroblocks of 4×8 pixels. As shown in FIG. 2D, in a fourth submacroblock type, one submacroblock is composed of four secondary submacroblocks of 4×4 pixels.
As shown in FIG. 1A, in the first macroblock type, one submacroblock of 16×16 pixels included in one macroblock is subjected to forward prediction (F), backward prediction (B), or bidirectional prediction (BI). Further, one submacroblock has one motion vector in one direction. Thus, one submacroblock has one or two motion vectors.
As shown in FIG. 1B, in the second macroblock type, the two submacroblocks of 16×8 pixels included in one macroblock is subjected to forward prediction, backward prediction, or bidirectional prediction, respectively. Further, one submacroblock has one motion vector in one direction. Thus, one submacroblock has two, three, or four motion vectors.
As shown in FIG. 1C, in the third macroblock type, the two submacroblocks of 8×16 pixels included in one macroblock are subjected to forward prediction (F), backward prediction (B), or bidirectional prediction (BI), respectively. Further, one submacroblock has one motion vector in one direction. Thus, one submacroblock has two, three, or four motion vectors.
As shown in FIG. 1D, in the fourth macroblock type, the four submacroblocks of 8×8 pixels included in one macroblock are subjected to forward prediction (F), backward prediction (B), or bidirectional prediction (BI), respectively. Further, although the secondary submacroblocks included in each submacroblock share a motion predicting direction, the motions of them are independently predicted. Further, each of the secondary submacroblocks included in one submacroblock has one motion vector in one direction. Thus, the number of the secondary submacroblocks included in one macroblock is 16 at the maximum, and when all of them are subjected to bidirectional prediction, one macroblock has 32 motion vectors.
Note that the submacroblock or the secondary submacroblock acting as a unit of prediction of motion is called a motion compensation block.
The number of bits of encoded data per one macroblock is as shown in FIG. 3. FIG. 3 will be briefly explained from a left side as follows.    (1) Intra-mode: 87 bits;    (2) Inter-mode, first macroblock type, forward prediction or backward prediction: 52 bits;    (3) Inter-mode, first macroblock type, bidirectional prediction: 83 bits    (4) Inter-mode, second or third macroblock type, two submacroblocks are predicted in forward direction or backward direction: 83 bits;    (5) Inter-mode, second or third macroblock type, one submacroblock is predicted in forward direction or backward direction, the other one submacroblock is predicted in bidirection: 114 bits;    (6) Inter-mode, second or third macroblock type, two submacroblocks are predicted in bidirection: 145 bits;    (7) Inter-mode, first to fourth macroblock types, all the motion compensation blocks are predicted in forward direction in P picture: 473 bits at the maximum: 473 bits;    (8) Inter-mode, fourth macroblock type, four submacroblocks are predicted in forward direction or backward direction: 473 bits;    (9) Inter-mode, fourth macroblock type, three submacroblocks are predicted in forward direction or backward direction, one submacroblock is predicted in bidirection: 582 bits;    (10) Inter-mode, fourth macroblock type, two submacroblocks are predicted in forward direction or backward direction, two submacroblocks are predicted in bidirection: 691 bits;    (11) Inter-mode, fourth macroblock type, one submacroblock is predicted in forward direction or backward direction, three submacroblocks are predicted in bidirection: 800 bits; and    (12) Inter-mode, fourth macroblock type, four submacroblocks are predicted in bidirection: 909 bits.
Accordingly, since the encoded data of H.264 per one macroblock is 909 bits at the maximum, it cannot be transmitted in a band of 256 bits per one macroblock prescribed by SMPTE 319-2000 standard.
Accordingly, to realize recoding while suppressing deterioration of image quality caused by the recoding as much as possible, any one of 1) a method of expanding a transmission band and 2) a method of effectively disposing data in a conventional transmission band.
Although the method of the item 1) is easy, when data is further multiplexed, it is contemplated that a disadvantage such as formation of a special visible pattern in an image, and the like. Further, it is also contemplated that compatibility with an MPEG-2 recoding device making use of a present system may be greatly broken.
As a result, the method of the item 2) cannot help being employed. When the method of the item 2) is employed, there is a possibility that an object of “suppressing deterioration of image quality as much as possible” cannot be achieved when data is disposed by an easy-going manner (that is, “easy-going data reduction”).
Accordingly, an object of the present invention is to provide an encoded data selection method and an encoded data selection device which can restrict encoded data per one macroblock transmitted from a decoding device to a recoding device to a predetermined value (for example, 256 bits) or less while suppressing the lack ratio of the amount of information of the encoded data.
Further, an object of the present invention is to provide an encoded data setting method and an encoded data setting method which can suppress deterioration of quality of a recoded image which is recoded by a recoding device making use of encoded data selected by the encoded data selection method and the encoded data selection device and shown by a bit stream.